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ABSTRACT:  Simulation  and  Modeling  for  Acquisition,  Requirements ,  and  Training  (SMART)  is  an  Army  activity  to 
bring  the  benefits  of  computer  simulation  to  all  phases  of  materiel  modernization  from  pre-Milestone  I  to  post- 
Milestone  III  analyses .  Given  the  extent  and  nature  of  (submodels  somehow  enveloped  in  this  totality,  key  issues  are 
raised  concerning  disparate  versus  redundant  submodels,  compatibility  and  granularity  of  metrics,  submodel 
integration,  maintenance  of proper  event  causality,  and  establishment  of  a  rational  sequence  for  model  instantiation.  A 
framework  developed  previously  for  vulnerability/lethality  (V/L)  studies  is  expanded .  This  framework  separates  metrics 
according  to  platform  component  integrity,  platform  aggregate  capabilities,  and  platform  battlefield  utilities .  In 
addition,  a  change  operator  that  can  modify  the  status  of  the  platform  component  capabilities  is  defined.  This  operator 
can  represent  not  only  ballistic  five-fire  events  (as  previously  exploited),  but  also  other  causes  of  component  breakage 
and  repair  as  required.  Also,  each  (metric)  class  is  connected  via  an  operator  that  reflects  intrinsic  platform  variables 
as  well  as  external  environmental  and  utility  factors.  Using  this  taxonomy,  existing  models  can  be  parsed  according  to 
the  particular  function(s)  they  perform.  The  ability  of  various  submodels  to  work  seamlessly  can  be  assessed,  and  the 
proper  sequencing  of  events  can  be  assured. 


1.  Introduction  &  Background 

1.1  What  is  SMART? 

Simulation  and  Modeling  for  Acquisition, 
Requirements,  and  Training,  or  SMART,  is  an  Army 
activity  in  which  Modeling  and  Simulation  (M&S)  are 
used  to  address  the  issues  of  system  development  and 
total  ownership  costs.  This  is  to  be  achieved  through 
the  collaborative  efforts  of  the  communities  supporting 
the  three  Army  functional  domains:  advanced  concepts 
and  requirements  (ACR),  research,  development,  and 
acquisition  (RDA),  and  training,  exercises,  and 
military  operations  (TEMO).  In  order  to  bring  the 
benefits  of  computer  simulation  to  all  phases  of 
materiel  modernization,  SMART  analysis  must  address 
two  major  issues.  The  first  is  the  military- analysis 
(MA)  or  system  architecture  part  of  the  problem.  This 
is  a  primary  objective  of  SMART  analysis.  However, 
since  the  context  of  M&S  is  supported  by  executable 
computer  code,  SMART  analysis  must  also  deal  with 
issues  of  computer  science  (CS),  or  the  technical 
architecture.  Table  I  lists  some  of  the  MA  issues  that 
confront  SMART.  Table  II  lists  a  few  of  the  issues 
that  shape  the  CS  component  of  SMART. 


CS  standards  both  enable  and  constrain.  It  is  important 
that  the  CS  tools  and  methods  do  justice  to  the  MA 
requirements  that  are  implicit  to  SMART  analysis. 
These  issues  apply  whether  the  simulation  is 
constructive  or  interactive.  However,  they  should 
follow  from  the  MA  requirements,  so  we  will  set  them 
aside  for  now  and  focus  on  the  MA  problem. 

Historically,  all  M&S  was  constructive.  The  emphasis 
was  on  good  engineering  fidelity.  The  models  tended 
to  be  narrow  in  scope,  and  frequently  the  CS  discipline 
and  computer  graphics  were  poor. 

In  the  distributed  interactive  simulation  (DIS)  world, 
CS  implementations  have  generally  been  of  higher 
quality,  including  interactive  graphics.  Often, 
however,  engineering  fidelity  has  been  poor  and  the 
scope  of  simulations  has  been  limited. 

It  is  importanfto  note  that  the  MA  world  is  changing 
rapidly.  The  number  of  analytical  elements  and 
challenges  is  growing.  These  include  the  need  to 
support  ever-increasing  complexity  in  the  area  of 
ballistic  mechanisms  and  interactions,  the  burgeoning 
area  of  communications  and  sensors,  the  need  to 
develop  truly  integrated  analyses,  and  the  requirement 


Table  I.  Some  Military  Analysis  (MA)  Elements  of  SMART 


Platform  Vulnerability 

System  Design 

Mission  Failure 

Battlefield  Effectiveness 

Crew  Casualties 

Probability  of  Hit 

Top  Speed 

Loss  of  Function 

Rate  of  Fire 

Time  to  Acquire 

Hit  Location 

Spall/Fire  Damage 

Probability  of  Detection 

Mean  Time  Between  Failures 

Battle  Damage  Repair  Time 

Cost  Effectiveness 

Table  II.  Some  Computer  Science  (CS)  Elements  of  SMART 


Communications  Protocols 

Ada 

High-Level  Architecture  (HLA) 

Open  Graphics  Library 

ASCII 

TCP/IP 

JMASS 

CORBA 

to  include  cost  as  an  independent  variable  (CAIV).  The 
superset  need  is  captured  in  the  DoD  focus  on  the 
Analysis  of  Alternatives  (AoA)  process,  which  has 
gained  overarching  importance.  In  addition  to  the  rapid 
advance  in  technologies  being  applied  to  military 
platforms,  the  operational  environment  is  also  in  rapid 
change.  New  missions  for  the  United  States  are  being 
defined;  they  are  typically  characterized  by  small  forces, 
forward  projection,  and  asymmetric  operations.  The 
need  is  great  for  lethality  projection  and  the 
minimization  of  friendly  casualties. 

The  principal  focus  of  this  paper  is  on  SMART  from  an 
analytical  perspective.  Throughout  the  proceeding 
words,  read  SMART  as  SMART  analysis. 

1.2  Changing  Times 

A  key  issue  is  what  constitutes  the  complete  military 
analysis  problem?  Is  there  an  overarching  framework  or 
context  within  which  the  elements  of  SMART  can  be 
described?  Given  the  pieces  of  SMART,  how  do  they  fit 
together?  What  levels  of  resolution  are  appropriate  for 
the  SMART  environment,  and  how  can  those  levels  be 
determined?  What  should  be  the  strategies  with  respect 
to  providing  lookup  (/. e.,  table-driven)  versus  predictive 
methods? 

In  the  next  section,  we  describe  an  example  of  SMART 
analysis  in  order  to  provide  an  overall  context  within 


which  to  fit  the  many  elements  or  submodels  and  to 
identify  gaps,  duplication,  and  submodel  connectivity. 

2.  SMART  Example:  BallisticVulnerability 

Ballistic  simulations,  both  vulnerability  as  well  as 
lethality,  have  always  been  fundamental  to  the 
assessment  of  military  fighting  vehicles.  We  review  the 
process  of  performing  a  live-fire  test  to  illustrate  how  the 
SMART  process  can  be  divided  into  three  major  pieces. 

Figure  1  shows  two  examples  of  a  live-fire  test.  On  the 
left  is  illustrated  a  shot  performed  during  the  Bradley 
Fighting  Vehicle  test  program;  on  the  right,  a  shot  made 
against  the  Apache-Longbow  helicopter. 

Typically,  such  shots  lead  to  particular  component 
damage  on  a  platform.  Loss  of  component  function  can 
lead  to  reduction  or  loss  of  certain  platform  capabilities. 
In  Fig.  2,  a  Soviet  T-62  tank  (on  the  left)  and  a  U.S. 
Apache  helicopter  (on  the  right)  demonstrate  climbing 
and  banking  maneuvers,  respectively.  Understanding 
both  preshot  (baseline)  and  postshot  (damage) 
performance  is  important  to  the  live-fire  process. 

Whether  working  at  nominal  specifications  or,  due  to 
damage,  at  some  reduced  level  of  capability,  a  military 
platform  must  be  viewed  in  the  context  of  its  mission 
utility  {i.e.,  its  ability  to  achieve  battlefield  success). 
Figure  3  illustrates  both  ground  and  air  platforms 
involved  in  campaign  activities. 
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Figure  1.  Ballistic  live-fire  encounters  with  Bradley  (left)  and  Apache-Longbow  (right). 


Figure  2.  Capability  testing  with  Soviet  T-62  tank  climbing  wall  (left) 
and  Apache  helicopter  banking  (right). 


Figure  3.  Two  kinds  of  military  missions,  one  involving  tanks  (left) 
and  the  other  involving  helicopters  (right). 
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3.  SMART  Analysis  Framework 

During  the  mid-1980s,  ballistic  Live-Fire  legislation 
[2]  brought  increased  attention  to  the  testing  and 
prediction  of  ballistic  live-fire  phenomenology.  An 
outgrowth  of  the  efforts  to  rationalize  the  comparison 
of  tests  and  prediction  was  the  Vulnerability/Lethality 
(V/L)  Taxonomy  [3-8].  This  framework  was 
developed  from  a  platform-centric  perspective  ( e.g ., 
the  numbering  system  is  related  to  the  platform  being 
described).  Later,  we  assert  that  this  analysis  strategy 
can  be  applied  to  multiple  platforms  engaged  both  in 
cooperative  and  adversarial  roles  (e.g.,  groups  of 
communications  platforms). 

3.1  Four  Classes  of  Metrics 

In  what  follows  we  employ  the  core  levels  of  the  V/L 
Taxonomy  as  described  in  Ref.  7.  This  concept 
characterizes  a  military  platform  (tank,  aircraft,  self- 
propelled  gun,  etc.)  in  terms  of  four  classes  of  metrics. 
The  classes  are  illustrated  in  Fig.  4  as  four  abstract 
levels  or  mathematical  spaces.  In  terms  of  the 
preceding  ballistic  live-fire  event,  Level  1]  represents 
the  complete  geometry/material  of  the  striking 
munition,  the  complete  geometry/material  of  the 
platform,  and  the  encounter  geometry  (e.g.,  hit  location 
and  kinematics);  this  information  can  be  represented 
by  a  vector  containing  the  initial  conditions  needed  to 
compute  the  resulting  damage  to  the  platform.  Level 
1]  is  shown  as  an  ellipse  filled  with  bullets  (•),  with 
each  bullet  representing  one  possible  encounter  vector. 

The  second  ellipse,  labeled  Level  2],  represents  the 
space  of  combinations  of  working  and  nonworking 
components  on  the  platform.  A  single  vector  at  this 
level  lists  the  outcome  status  of  each  platform 
component. 

The  third  ellipse,  labeled  Level  3],  represents  the 
measurable  capabilities  of  the  platform.  For  a  fighting 
vehicle,  this  would  typically  consist  of  its  abilities  to 
move,  acquire,  communicate,  and  fire  its  guns.  Again, 
each  bullet  represents  a  vector  that  describes  some 
particular  state  of  the  platform  capabilities.  Level  3] 
metrics  can  be  considered  measures-of-performance 
[MoPs]. 

The  fourth  ellipse,  labeled  Level  4],  represents  the 
mission  utility.  In  effect,  this  metric  indicates  that  the 
platform  is  either  able  or  unable  to  meet  the  current 
requirements  of  the  military  campaign.  Level  4) 
metrics  can  be  considered  measures-of-effectiveness 
[MoEs]. 


In  Table  III,  we  list  each  level,  indicate  that  a  vector 
associated  with  that  level  has  a  corresponding 
subscript,  and  then  describe  the  information  typically 
represented  by  the  vector.  Except  for  the  Level  4] 
vector,  v4,  each  of  the  vectors  in  Levels  1]  through  3] 
is  observable,  measurable,  and  testable.  This  is 
important  for  supporting  verification,  validation,  and 
accreditation  (VV&A)  activities  that  necessarily 
support  these  abstractions.  In  terms  of  our  ballistic 
example,  a  vector  at  Level  1],  Vj,  defines  the  threat, 
the  platform  under  test,  and  the  kinematics  of  delivery. 
The  results  of  the  live-fire  test  generally  lead  to 
damage  to  particular  components  on  the  platform.  The 
complete  status  of  each  component  ( i.e .,  killed  [■], 
not-killed  [□])  would  be  given  by  vector  v2.  Were  the 
vehicle  to  be  tested  with  movement,  gun  firing,  etc., 
the  capabilities  of  the  platform  would  be  represented 
by  the  vector  v3.  Platform  utility,  typically  not 

measurable,  would  be  represented  by  the  vector  v4. 
The  critical  issue  of  military  utility  is  discussed  later. 
For  more  on  the  properties  of  these  vectors,  see  Ref.  7. 

3.2  Three  Classes  of  Operators 

How  are  these  vector  levels  linked?  Mathematically, 
they  can  be  linked  by  operators  or  transformation 
processes  that  take  a  vector  at  Level  n]  and  map  it  to  a 
vector  at  Level  n+1].  The  mathematical  operators 
described  are  at  the  heart  of  the  SMART  analysis 
process.  Alternatively,  the  operators  can  be  viewed  as 
tests  or  experiments  performed  in  the  field.  The 
linkage  or  corroboration  between  the  results  of 
analytical  operators  and  field  tests  provide  the  basis  for 
all  rigorous  W&A  activities.  Figure  4  also  shows 
how  the  four  levels  of  metrics  are  linked. 

The  four  levels  are  connected  abstractly  by  operators 
written  as  On  n.  This  notation  leads  to  the  convention 

Vq  =  °p,q{Vp}-  (1) 

Since  the  operators  can  only  connect  sequential  levels, 

Vi =  °p.p+i  { vp (2) 

The  live-fire  tests  illustrated  in  Fig.  1  are  represented 
by  the  Ol  2  operator.  The  test  processes  shown  in  Fig. 

2  are  represented  by  the  023  operator.  The  mission 
exercises  illustrated  in  Fig.  3  are  represented  by  the 
03  4  operator. 
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Oj  2  Operator 

Risk-Platform  Initial 
Conditions  to  Platform 
Components  Change 


-< 


023  Operator 

Component  Status  State  ^ 
to  Platform 

Measures-of-Capability 


034  Operator 
Platform 

Measures-of-Capability 
to  Platform  Mission  Utility 


Level  1] 

Risk-Repair 
Input  /Initial 
Conditions 


Level  21 
Platform 
Component 
Configuration/ 
Status 


Level  31 

Platform 

Aggregate 

Capabilities 


Level  4J 
Platform 
Mission 
Utility 


Risk  Factors 


Military 

Operations 

Context 


X  \ 

•  Doctrine 

\  \ 

•  Tactics 

\ 

•  Leadership 

▼^c0/7  ^ - 

*  Organization 

•  Materiel 

•  Scenario 

*  Terrain 

(Global 

- - 

Variables) 

Figure  4.  V/L  Taxonomy  illustrated  via  a  mapping  abstraction.  The  ellipses  represent  mathematical  spaces. 
The  bullets  (•)  contained  within  the  spaces  represent  vectors.  The  connecting  arrows  represent  operators  that  map 
a  vector  at  one  level  to  a  vector  at  the  next  sequential  level.  On  the  left,  the  descriptors  for  the  various  levels  and 
operators  are  listed.  On  the  right,  a  box  labeled  Military  Operations  Context  provides  descriptors  of  the  external 
military  environment  (mission,  terrain,  threats,  etc.)  within  which  the  platform  must  perform. 


Table  III.  Composition  of  Vectors  at  Each  Level  or  Class 


Level 

Vector 

Vector  Descriptor 

ii 

vi 

Platform  Geometry  &  Material, 

Risk/Repair  Geometry  &  Material, 

Encounter  Geometry 

21 

v2 

Status  of  Platform:  Le ,  Working  (□) 
and  Nonworking  (■)  Components 

31 

v3 

Platform  Capability:  e.g.  Mobility,  Firepower, 
and  Communication  Functions 

41 

v4 

Platform  Utility:  e.g.  Does  Platform  Survive? 

(from  Level  2]), 

Can  Platform  Perform  Specific 
Mission- Tasks? 

5 


In  order  to  perform  a  live-fire  shot,  all  of  the  target 
system  has  to  be  represented  in  the  test.  To  model 
live-fire  accurately,  the  three-dimensional  geometry 
must  be  described  to  high  fidelity  as  well.  Figure  5 
shows  exterior  and  interior  images  [9]  of  a  Soviet  T-62 
solid  geometric  model  tank  built  with  the  Army¬ 
generated  solid-geometry  package  BRL-CAD®  [10]. 
This  geometry  is  represented  by  the  vector,  Vj. 

To  the  right  of  the  four  ellipses  is  found  a  box  labeled 
Military  Operations  Context  (MOC).  This  construct 
has  been  added  to  recognize  that  each  of  the  operators 
takes  input  from  the  operations  context  in  which  a 
platform  performs.  The  MOC  defines  the  doctrine, 
tactics,  leadership,  materiel,  scenario,  terrain,  weather- 
all  of  the  factors  external  to  the  platform  itself.  For 
example,  during  a  live-fire  test,  the  volatility  of 
ammunition  that  may  be  ignited  is  dependent  on  the 
ambient  temperature  for  a  given  day-hence,  the 
context  data  feed  to  the  012  mapper.  Similarly,  with 
the  02  3  capability  mapper,  the  ability  of  a  platform  to 

move  or  acquire  is  a  function  of  the  terrain  and 
weather  variables-  hence  the  .context  data  connection 
to  this  mapper.  Finally,  the  MOC  clearly  defines  the 
mission  activities  or  tasks  that  the  platform  will  have 
to  perform  in  order  to  achieve  mission  success.  The 
required  task  levels  are  fed  to  the  03  4  operator  from 
the  MOC  as  well.  In  effect,  a  significant  portion  of  the 
MOC  is  defined  by  the  Army  DTLOMS  (doctrine, 
training,  leader  development,  organization,  materiel, 
and  soldier  structure).  Importantly,  the  03  4  operator 
maps  platform  capabilities  to  mission  outcomes  by 
bringing  to  the  platform  a  sequence  of  mission  tasks 
(as  well  as  mission  risks).  Mission  outcomes  are  the 
basis  for  defining  mission  success.  This  particular 
mapping  may  represent  the  greatest  analytic  challenge 
facing  the  Army  today.  For  a  notional  example  on 
how  mission  success  is  determined  by  a  platform 
current  versus  required  capability,  see  Ref  7,  Fig.  4. 

3.3  Extant  Codes 

The  previously  listed  abstractions  are  not  simply 
notional.  They  have  been  implemented  in  code  and 
used  to  describe  complex  ballistic  interactions, 
compute  resulting  damage  and  related  capability 
diminishment.  The  specific  production  code  consists 
of  a  family  of  BRL-CAD  geometry  tools  [10]  and  a 
suite  of  C-based  vulnerability  codes  supported  by  a 
special  portable  environment.  Called  Modular  UNIX- 
Based  Vulnerability  Estimation  Suite  (MUVES)  [11], 
this  single  supporting  structure  is  used  to  compute  tank 
direct-fire  interactions,  anti-air  encounters,  and 
indirect-fire  artillery  and  bomblet  events.  These 


capabilities  are  also  provided  for  personnel 
vulnerability  evaluation  by  a  model  called  Operational 
Requirements  for  Casualty  Assessment  (ORCA)  [12], 
which  uses  the  same  taxonomy. 

3.4  Generalization  of  the  Ox  2  Operator 

Though  originally  conceived  as  a  damage  operator  for 
representing  ballistic  phenomenology,  the  Oj  2 

Operator  has  been  generalized  across  a  range  of 
damage  and  risk  mechanisms  including  the  description 
of  reliability,  availability  and  maintainability  (RAM) 

[13]  as  well  as  electronic  warfare  and  chemical  threats 

[14] .  In  addition  to  causing  damage,  the  Oj  2  Operator 

can  also  represent  repair  or  fix  operations  and,  hence, 
supply  or  replenishment.  A  partial  list  of  actions  that 
fit  the  Oj  2  framework  is  given  in  Table  IV. 

3.5  Event/Time-Stepping  the  Taxonomy 

When  considered  in  the  context  of  a  mission,  the 
Taxonomy  framework  illustrated  in  Fig.  4  should  be 
considered  dynamic.  That  is,  as  a  platform  proceeds 
through  time,  the  MOC  a)  causes  degradation  to  the 
system  (component)  infrastructure  and  b)  challenges 
the  capabilities  of  the  platform  with  a  sequence  of 
tasks.  This  notion  is  a  compact,  yet  detailed  way  of 
thinking  about  mission  activities.  The  metrics  of  the 
four  levels  and  the  connecting  operators  are,  in  fact, 
the  pieces  of  SMART  analysis  and  how  the  pieces 
relate  to  each  other. 


4.  Discussion  of  Framework 

4.1  Three  Kinds  of  Operations 

In  the  context  of  SMART  analysis,  we  can  see  that 
there  are  three  major  classes  of  modeling  and  testing 
that,  though  different  in  detail,  reoccur  in  virtually 
every  study.  They  are: 

1]  How  risk/repair  factors  change  the  platform 

microstructure, 

2]  How  platform  micro  structure  leads  to 

platform  macroperformance,  and, 

3]  How  platform  macroperformance  leads  to 

military  success/failure. 

It  is  our  conjecture  that  the  elements  of  SMART  can  be 
broken  down'  and  placed  in  one  of  these  three 
categories.  This  observation  has  important 
ramifications  that  include  assessing  the  completeness 
of  operations  research  (OR)  studies,  the  sharing  of 
various  tools/databases,  and  the  achievement  of  global 
modeling  coverage. 
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Figure  5.  BRL-CAD®  solid  geometry  target  description; 
outside  of  foreign  tank  (left),  inside  (right)  [from  Ref.  9]. 


Table  IV.  Generalizing  the  Oj  2  Ballistic  Damage  Operator  to  the  Oj  2  Change  Operator 


tOuasM  Permanent  Damage 

Temporarv  Damage 

Comnonent  Fix/SuDDlv 

Ballistic 

Electronic  Jamming 

Battle  Damage  Repair 

Chemical 

Cosite  Interference 

Logistics  Resupply 

Directed  Energy 

Sleep  * 

High-Power  Laser 

Nuclear 

Logistics  Burdens  (e.g.,  Fuel,  Ammo) 

Reliability 

Physics  of  Failure 

Fair  Wear  &  Tear 

Fatigue* 

Heat  Stress* 

*  Personnel  Related 


4.2  Two  Major  Links  to  MA 

Military  platforms  moving  through  missions  are 
confronted  with  two  classes  of  actions: 

1]  Components  are  degraded  /upgraded  by 

risk/repair  factors,  and 

2]  Platform  capabilities  are  tested  for 

mission  adequacy. 

As  a  mission  progresses,  various  risks  erode  the 
fidelity  of  the  components.  At  each  task  encounter,  the 
sufficiency  of  capability  is  challenged.  As  damage 
increases,  capabilities  diminish  and  continue  to  be 
(mission)  challenged,  etc . 


4.3  Multivariate,  Nonlinear,  &  Stochastic  Mappings 

The  operators  which  populate  each  of  the  three  classes 
can  be  extremely  complicated.  In  general,  the 
mapping  operations  are  multivariate  (i.e.,  depend  on 
many  factors),  nonlinear,  and  are  often  stochastic.  For 
a  discussion  of  these  effects  on  the  mapping  results, 
see  Ref.  7. 
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4.4  Metric  Variations:  Level  2] 

When  risks  are  played  at  Level  1],  three  general 
outcomes  can  occur  at  Level  2].  They  include: 


1]  no  damage, 

2]  intermediate  damage,  and 

3]  complete/catastrophic  damage. 

Assessment  of  the  last  outcome  is  obvious.  However, 
there  are  considerable  challenges  in  dealing  with  the 
case  of  intermediate  damage  outcome  which  leads  to 
reduced  capability  and  even  defining  in  a  meaningful 
way  what  constitutes  full  mission  utility.  The  current 
practice  of  normalizing  full  mission  utility  to  a 
baseline  of  1.0  is  not  particularly  illuminating  either  in 
an  absolute  sense  or  in  a  comparative  one  (/.  e. ,  one 
platform  utility  versus  another). 

Repair  activities  may  result  in  the  full  restoration  of 
the  platform  to  top  condition  or,  in  the  case  of 
expedient  repair,  to  only  some  of  the  components 
being  restored.  This  results  in  a  system  with  utility 
that  is  mission  dependent  as  well. 

4.5  Aggregation  of  Damage 

To  evaluate  properly  a  particular  risk  or  repair 
interaction,  the  result  of  all  prior  interactions  must  be 
accounted  for;  thus,  the  next  event  sees  the  platform  in 
its  current,  not  pristine,  condition.  The  majority  of 
simulations  assume  risk/repair  interactions  with 
“pristine”  platforms,  whereas  in  extended  missions, 
this  condition  represents  the  exception  rather  than  the 
rule.  This  expediency  can  lead  to  erroneous 
assessments. 

4.6  Dynamic  Geometry 

It  is  important  to  note  that  the  ability  to  exercise 
predictive  or  physics-based  models  in  this  SMART 
context  is  based  in  part  on  the  ability  to  interrogate,  as 
needed,  3-D  solid  geometry.  Because  of  the  diverse 
numbers  of  risk/repair  encounter  conditions,  it  is  not 
possible  to  precompute  all  of  the  outcomes  needed  a 
priori .  Nevertheless,  it  is  the  practice  of  the  force-on- 
force  community  to  use  table  lookup  procedures,  rather 
than  computing  damage  or  capability  metrics  as 
needed,  “on  the  fly.” 

A  yet  more  advanced  capability,  and  one  required  in 
future  analysis  environments,  is  dynamic  geometry. 
This  is  a  capability  in  which  the  results  of  a  mission 
event  can  be  used  to  modify  the  geometric  data 
originally  used  as  input.  This  is  important  for  the 
study  of  platforms  whose  utility  must  be  examined 
over  extended  periods  of  time  during  which  geometry 
and/or  material  undergo  critical  modification.  Such 
changes  alter  the  character  of  signatures,  performance, 
and  protection,  not  to  mention  similar  issues  with  the 


MOC  ( e.g changes  in  terrain  features  due  to  ballistic 
impacts,  vehicle  tracks).  BRL-CAD  has  utilities 
incorporated  into  the  environment  so  that  intermediate 
computations  can  be  used  to  modify  geometric  source 
files. 

4.7  Metric  Variations:  Level  3| 

The  effect  of  a  platform  with  some,  but  not  all, 
components  functional,  is  that  it  may  lead  to 
reductions  in  various  capabilities  at  Level  3].  This 
raises  a  difficult  problem  in  the  assessment  of  military 
utility  at  Level  4].  This  problem  was  recognized  in  the 
1950s  by  workers  at  the  U.S.  Army  Ballistic  Research 
Laboratory  (BRL).  They  concluded  that  the  military 
utility  of  a  platform  with  degraded  capabilities  depends 
on  the  mission  requirements  [15].  They  developed 
ballistic  metrics  that  estimated  a  (normalized)  utility 
averaged  over  all  missions.  For  more  than  40  years 
direct-fire  ballistic  metrics  have  been  computed  so  as 
to  use  (Level  4])  average  utility  metrics,  rather  than 
capability  metrics  at  Level  3].  Exploratory  work  has 
been  performed  for  a  number  of  years  to  understand 
the  benefits  and  burdens  of  higher  resolution  methods 
[16,  17]. 

4.8  Multi-Platform  Analyses 

This  process  can  also  be  applied  to  “systems-of- 
systems.”  For  analyses  in  which  multiple  platforms 
are  being  played,  each  platform  requires  its  own  four- 
level  vector  description.  This  approach  is  relevant  to 
analyses  of  C4I  systems.  The  O,  2  map  (evaluating 
jammers,  worms,  etc.)  is  worked  for  each  cooperative 
platform.  An  appropriate  02j  map  is  then  computed 
across  the  gamut  of  linked  systems.  After  aggregate 
MoPs  are  computed,  the  034  utility  mapping  is 

performed  for  the  joint  platforms  in  mission  context. 

5.  Taking  Stock:  Current  Practice 

Looking  at  today’s  SMART  simulations,  a  number  of 
generalizations  can  be  made. 

5.1  Risk/Repair  Analyses 

The  models  of  this  class  typically  demonstrate  the  best 
physics  and  engineering.  Since  the  scope  of  the 
problem  is  often  smaller,  limited  resources  can  be 
concentrated  on  a  more  narrow  set  of  issues.  Often, 
studies  of  this  class  will  look  at  failure  rates  ( e.g .,  due 
to  one  or  another  phenomenology),  but  to  the 


detriment  of  overall  significance,  provide  no  linkage  to 
platform  capability  or  military  utility. 

5.2  Engineering  Performance  Analyses 

Engineering  mappings  are  among  the  most 
sophisticated  in  the  SMART  analysis  world.  They 
frequently  represent  some  of  the  best  predictive 
capabilities.  All  too  often,  however,  there  is  no  clear 
connection  between  predicted  engineering 
performance  at  Level  3]  and  utility  at  Level  4].  In 
addition,  the  02?3  engineering  mappings  are  frequently 
performed  assuming  all  coniponents  are  fully 
operational.  Thus,  there  may  be  no  ability  to  assess 
capability  under  arbitrary  levels  of  component  damage 
and,  hence,  infer  the  likelihood  of  graceful  degradation 
for  increasing  damage.  The  link  from  risk/repair  to 
component  status  (Level  2])  is  seldom  made. 

5.3  OR  Analyses 

Investigations  into  effectiveness  are  normally 
supported  through  wargames.  Historically  oriented 
toward  V/L  computations,  V/L'  metrics  are  drawn  from 
table  look-up. 

In  addition  to  the  aforementioned  problems  of 
computing  V/L  estimates  for  an  expected  military 
utility,  the  kill  metrics  are  further  compromised  by 
computing  the  average  mobility  and  firepower  kills  at 
a  particular  hit  location.  These  are  combined  into  a 
single  number,  weighted  by  a  bivariate  hit  distribution. 
Next,  these  distributions  are  weighted  by  a  cardioid  hit 
distribution,  weighted  to  strikes  from  the  front.  There 
normally  is  an  assumption  that  the  encounter  is  for  a 
first  hit.  Thus,  there  is  no  proper  aggregation  of 
damage  at  Level  2],  no  notion  of  prior  events. 

As  noted  in  Section  4.7,  the  methods  needed  to  avoid 
premature  averaging  have  been  established  [13,  16, 
17].  The  tools  are  in  hand  to  calculate  damage,  even 
when  prior  damage  exists.  A  significant  issue  is  what 
are  the  Level  4]  utility  metrics  or  MoEs. 

In  general,  the  relationship  between  the  platform 
component  status  and  platform  capabilities  is  not 
modeled  explicitly,  and  the  relationship  between  the 
detailed  platform  capabilities  and  mission  utility  is  not 
played.  Since  there  is  no  detailed  description  of 
platform  geometry,  it  is  not  possible  to  perform 
risk/repair  component  erosion. 

Thus,  there  is  no  functional  thread  in  the  OR 
assessment  process  from  operational  risks/platform 


repairs  (through  performance)  to  OR  utility.  This 
makes  the  proper  assessment  of  technology  tradeoffs 
and  system  assessments  very  problematic  and  results 
in  a  fundamental  cognitive  disconnect  between  the 
U.S.  Army  Materiel  Command  (AMC)  as  a  provider  of 
technology  (read  performance)  and  the  U.S.  Training 
and  Doctrine  Command  (TRADOC)  as  a  customer  of 
military  utility.  In  other  words,  in  weapons  analyses, 
analysts  often  fail  to  establish  a  clear  relationship 
between  MoPs  and  MoEs.  This  reality  has  not  been 
lost  on  the  Office  of  Secretary  of  Defense/Program 
Analysis  and  Evaluation  (OSD/P A&E).  A  number  of 
recent  Army  systems  have  fallen  victim  to  the  apparent 
inability  to  relate  system  performance  to  mission 
effectiveness. 

5.4  What  Are  the  Right  MoEs? 

Historically,  force-on-force  modeling  has  concentrated 
on  loss  exchange  ratios  (LERs),  trading  projected 
lethality  against  received  vulnerability.  Current  force- 
on- force  modeling  practices  can  be  traced  to  the  1960s 
with  even  earlier  roots  when  warfare  was  structured 
around  massed  forces  on  the  European  plains.  Central 
limit  tendencies  could  be  invoked  given  the  large 
numbers  at  play.  Today,  the  international  situation  is 
much  changed.  It  is  widely  acknowledged  that  there 
will  be  no  more  Desert  Storms  where  essentially 
symmetric  (mirror  image)  forces  come  into  play.  It  is 
unlikely  future  U.S.  opponents  will  attempt  to  fight  on 
“common  ground.”  Asymmetrical  warfare  will  be 
more  likely. 

Consider  the  U.S.  actions  in  Mogadishu.  Well 
documented  in  Black  Hawk  Down  [18],  on  the  day  in 
question,  U.S.  forces  killed  more  than  1000  natives  at  a 
cost  of  18  U.S.  troops-by  classic  measures,  a  U.S. 
numerical  victory,  but  hardly  considered  a  military 
triumph  by  most  Americans. 

Consider  the  U.S.  actions  in  Kosovo.  The  U.S.  Air 
Force  was  sufficiently  preoccupied  with  pilot 
casualties  that  air  missions  were  prosecuted  at  altitudes 
for  which  precise  ordnance  delivery  became  difficult. 

Beyond  the  relevancy  of  LERs  as  a  primary  utility 
figure  of  merit,  system  costs  have  entered  the  picture 
in  a  significant  way.  All  of  this  comes  together  under 
DoD’s  current  thrust  toward  AoAs.  If  a  set  of 
competing  platforms  is  going  to  be  traded  off,  what  is 
the  appropriate  figure-of-merit  by  which  to  evaluate 
the  trade? 
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5.5  Summary:  Current  Practice 

Many  elements  of  SMART  analysis  exist  within  the 
M&S  community.  Seldom,  however,  are  the  elements 
of  the  three  classes  of  computation  (i.e.  risk/repair, 
engineering  performance,  and  mission  effectiveness) 
linked  together  so  as  to  support  the  entire  decision 
process. 


6.  Rethinking  the  Process 

Since  the  introduction  of  the  Taxonomy  in  Fig.  4,  we 
have  been  considering  the  process  in  a  time  forward  or 
causal  sense.  Such  a  procedure  has  the  potential  for 
mapping  to  an  outcome  space  that  is  both  accurate  and 
irrelevant.  This  issue  is  raised  when  one  considers  just 
where  the  beginning  point  is  when  setting  up  a 
SMART  analysis  environment. 

6.1  Starting  at  the  End 

Even  if  a  complete  set  of  computer  codes  were  written 
and  fully  integrated,  there  would  still  be  the  task  of 
setting  up  the  key  inputs  to  the  codes.  But  where  to 
begin?  We  suggest  beginning  with  Level  4],  Mission 
Outcome  Space,  and  then  working  back  to  Level  1J. 

We  turn  our  attention  to  Fig.  6  in  which  the  Taxonomy 
has  been  reproduced  with  a  few  additions.  At  Level 
4],  a  number  1  has  been  placed  to  the  left  of  the  ellipse. 
The  first  step  in  the  instantiation  of  this  process  should 
be  a  thorough  discussion  of  the  set  of  mission 
outcomes  as  implied  by  the  vectors  of  Level  4] 
followed  by  a  decision  of  just  what  subset  of  those 
outcomes  is  acceptable  as  successful  missions. 
Notionally,  we  have  sorted  the  acceptable  outcomes  of 
mission  space  into  the  circle  shown  within  the  ellipse. 
Deciding  just  what  constitute  the  mission  MoEs  may 
be  one  of  the  most  challenging  aspects  of  the  SMART 
analysis  process.  However,  if  this  first  step  is  not 
performed  well,  then  there  is  no  basis  for  technical 
performance  goals  or  cost  tradeoffs. 

Given  mission  success  is  established  at  Level  4],  the 
next  step  is  to  understand  what  set  of  performance 
metrics  at  Level  3|  will  map  to  the  “circle-of-success” 
at  Level  4]  in  the  context  of  the  MOC.  In  this  process, 
the  nature  and  height  of  the  mission  “hurdles”  (as  in 
track  &  field)  are  specified  and  analyzed.  If  the 
mission  performance  requirements  are  identified  and 
set  properly,  in  effect  the  03  4  mapper  for  the  subject 
mission  is  defined.  This  completes  Step  2.  At  this 
point,  the  nature  and  level-of-performance  metrics  at 


Level  3]  needed  for  success  at  Level  4]  can  be  defined. 
If  this  process  were  to  be  accomplished  truly 
agnostically,  there  fnight  exist  a  diverse  population  of 
capability  vectors  that  might  be  reflected  in 
substantially  different  technical  solutions.  Step  3  is 
complete  when  the  Level  3]  “circle-of-success”  is 
defined.  With  Step  3  complete,  it  is  only  then 
appropriate  to  seek  specific  technical  solutions  to  a 
military  mission  need.  If  this  framework  is  correct,  it 
says  that  military  mission  success  depends  on 
capabilities,  not  technology  (which  is  a  Level  2] 
metric)  per  se.  Given  a  performance  envelope  defined 
at  Level  3],  system  designers  can  next  postulate 
materiel  solutions  at  Level  2],  in  Step  4.  Then  through 
engineering  analysis,  the  02  3  mapping  operation  can 

be  estimated  via  modeling,  or,  if  the  platform  is  built, 
defined  by  test.  This  is  Step  5  and  the  point  at  which 
engineering  analysis  confirms  that  a  potential  technical 
solution  exits  capable  of  meeting  the  performance 
requirements  defined  at  Level  31.  Also,  those 
components  critical  to  achieving  success  at  Level  3] 
can  be  identified  and  sorted  into  the  “circle-of- 
success”  at  Level  2J.  If  analyses  and/or  tests  confirm 
that  the  performance  is  as  expected  and  adequate,  cost 
figures  can  be  assigned  in  Step  6.  At  this  point,  an 
apparent  solution  has  been  identified,  subject  to  the 
assessment  of  risks.  In  Step  7,  the  MOC  is  queried  to 
identify  all  military  threats  (bullets,  high-power 
microwaves,  etc.)  as  well  as  the  operational  context, 
which  speaks  to  wear  and  tear  on  equipment  and 
people.  Once  identified,  these  risk  factors  are  mapped 
from  Level  1]  to  Level  2]  to  examine  whether  any  of 
the  critical  components  shown  within  the  “circle-of- 
success”  is  threatened.  If  so,  platform  performance 
and,  hence,  mission  success  are  put  at  risk.  Step  8  is 
the  establishment  of  the  appropriate  Oj  2  change 
operators  as  driven  by  the  MOC  and  inherent  platform 
susceptibilities  to  failure/breakage.  Step  9  indicates 
that  this  process  can  be  run  repetitively  as  the  MOC 
defines  a  mission  script  and  drives  the  four-level  state 
vector  represented  by  the  Taxonomy  through  time 
from  the  beginning  to  the  end  of  the  mission.  Given 
adequate  exercise  of  this  framework  over  reasonable 
sample  sizes,  an  improved  linkage  could  be  established 
between  technology,  performance,  and  utility  for  a 
wide  variety  of  scenarios  as  defined  by  some  set  of 
MOCs. 

This  framework , has  been  used  recently  to  suggest  a 
procedure  for  planning  live-fire  test  strategies  with  a 
view  to  cost-effective  practices  [19]. 
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Level  2]  4,6 


Level  3] 


Level  4] 


Military 

Operations 

Context 


•  Doctrine 

•  Tactics 

•  Leadership 
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•  Materiel 

•  Scenario 

•  Terrain 
(Global 

Variables) 


Subspace  of  vectors  that 
map  to  a  “success  subspace” 
at  the  next  Level 


Fig.  6.  V/L  Taxonomy  used  to  order  the  process  of  framework  build.  To  instantiate  the  SMART  analysis  process, 
the  end  point  is  established  first;  then  mapping  and  metrics  worked  toward  the  beginning.  The  circles  represent  the 
level  subspaces  which  are  required  to  lead,  ultimately,  to  mission  success. 


6.2  Is  This  Practice  New? 

How  new  is  this  practice?  Actually,  competent 
analysts  have  always  known  that  to  construct  a 
satisfactory  model,  they  need  to  examine  the  desired 
end  result  and  infer  their  way  back  to  the  required  prior 
conditions.  This  approach  has,  in  fact,  been  practiced 
by  the  Air  Force  analysis  community.  In  the  1980s, 
LTG(R)  Glenn  Kent  established  a  procedure  called 
“Mission-to-Task  Process”  [20]  in  which,  in  effect,  a 
mapping  process  is  established  to  make  clear  the 
relationship  between  Level  3],  performance,  and  Level 
4],  military  utility.  This  is  the  critical  problem  of 
relating  MoPs  to  MoEs.  More  recently,  a  hierarchical 
decision  architecture  using  fuzzy  set  theory  has  been 
applied  to  the  Mission-to-Task  problem  [21]. 

The  Army  actually  has  an  established  history  of  using 
Mission-to-Task  methods  to  establish  a  framework 
within  which  human  performance  can  be  evaluated 
[22-26].  However,  there  are  apparently  few  examples 


within  the  Army  for  the  application  of  these  techniques 
to  purely  nonhuman,  materiel  analysis. 

By  the  Mission-to-Task  Process,  in  effect  the  034 
mapping  is  generated  in  the  context  of  a  specific 
mission  where  the  Level  4]  utility  vector  is  based  on  a 
combination  of  Level  2]  metrics  (loss  of  critical 
components,  including  crew),  resulting  in  subthreshold 
capabilities  vis-a-vis  task  requirements.  The  task 
requirements  are  defined  by  the  MOC.  Thus,  a  Level 
4]  can  be  written  functionally  as: 

v4  =  f(v2,v3,MOC).  (3) 

It  is  probably  appropriate  that  all  MoEs  be  defined  in 
terms  of  MoPs,  which  can  be  either  component 
performance  metrics  or  platform  performance  metrics. 
Although  challenging,  there  is  increasing  evidence  that 
OR  analysts  are 'increasingly  moving  in  this  direction, 
particularly  as  the  AoA  process  requires 
nonmetaphysical  measures  to  be  demonstrated  in  the 
test  arena  while  simultaneously  making  the  case  for 
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military  worth,  which  in  the  main  is  probably  not 
directly  observable! 


7.  Summary  &  Conclusions 

We  have  proposed  a  structure  for  SMART  analysis 
that: 

•  Shows  connected  performance  and  operational 
pieces, 

•  Shows  four  distinct  classes  of  metrics  connected 
by  three  distinct  classes  of  operators, 

•  Shows  specific  connections  between  the  military 
operational  context  and  various  platform  metrics 
and  operators. 

This  illustrates  a  widespread  class  of  military  OR 
problems,  typified  by  the  following  key  steps: 

1.  Define  baseline  system  configuration, 

2.  Estimate  system  performance, 

3.  Estimate  system  effectiveness, 

4.  Account  for  change  in  system  configuration  due 
to  threats,  etc.,  and 

5.  Go  to  2,  etc. 

An  adequate  SMART  analysis  requires  all  pieces  to  be 
properly  constructed,  linked,  and  integrated.  The 
cornerstone  of  SMART  should  be  the  process  of 
relating  platform  capabilities  to  desired  mission 
outcomes.  In  SMART,  the  whole  is  greater  than  the 
sum  of  its  parts  when  developed  with  highly  resolved 
operational  contexts.  The  03  4  Mission  Operator  and 

Level  4]  utility  status  reflect  our  ability  to  understand 
what  should  become  both  the  beginning  and  the  end  of 
our  analysis  cycles,  operational  success. 

This  process  says  that  mission  outcomes  (including 
success)  are  related  to  platform  capabilities,  not 
directly  to  technology.  Thus,  platform  capability 
tradeoffs  should  be  established  in  the  context  of 
mission  performance  first,  and  then  technology  should 
be  perused  and  exploited  for  opportunities  to  meet 
those  capabilities  (read  requirements),  subject  to 
appropriate  constraints. 

We  gain  insight  into: 

•  The  manner  in  which  the  Operational,  Technical, 
and  Intelligence  communities  need  to  work  together, 
and 

•  The  implications  for  building  the  Army  technology 
program. 


The  framework  can  be  used  to  parse  various  M&S 
models  to  see  into  what  portion  of  SMART  analysis 
they  fit.  It  can  be  used  to  identify  redundancies  in 
some  areas  and  coverage  gaps  in  others. 

The  framework  can  be  used  to  set  interface  and 
architectural  standards  (per  various  levels),  make  clear 
how  algorithms  are  employed,  highlight  code  gaps, 
and  provide  critical  global  connectivity.  It  is  at  this 
point  that  the  many  computer  science  metrics  and 
connectivity  tools  can  be  best  employed. 

This  decomposition  shows  that  a  large  number  of 
destructive  (e.g.,  ballistic,  chemical,  nuclear,  logistics 
burden,  and  reliability)  and  constructive  ( e.g .,  battle 
damage/expedient  repair,  and  product  improvement 
process)  operators  require  similar  inputs,  perform 
similar  tasks,  and  modulate  the  status  of  the  same 
components. 

The  framework  illustrates  that  damage  and 
repair/upgrade  occur  at  Level  2].  Multiple  instances  of 
damage/repair  must  aggregate  at  Level  2],  not,  as  is 
often  done,  by  manipulating  Level  4]  metrics.  This 
approach  can  be  used  to  analyze  “Systems-of- 
Systems”  (e.g.,  networked  communications  platforms). 

Engineering  issues  focus  on  the  023  operator.  This 
operator  takes  Level  2]  metrics  and  operational  context 
data  and  must  compute  all  capability  metrics  relevant, 
ultimately,  to  mission  success.  These  include  weights, 
moments-of-inertia,  movement,  all  sensors, 
communications,  guns/missiles  capabilities,  etc.  A 
large  number  of  required  engineering  operators  require 
similar  inputs,  perform  similar  tasks,  and  modulate 
identical  capability  status  metrics.  This  process 
illustrates  the  importance  of  global  variables  sourced 
in  the  MOC  and  coupled  to  various  levels  and 
operators. 

Given  a  prescribed  MOC  (threats,  mission,  etc.),  an 
inverse  information  threading  can  be  established  that 
points  through  critical  performance  factors,  to 
supporting  critical  components,  and,  finally,  to  any 
factors  that  put  them  at  risk.  These  relationships  can 
be  used  to  prioritize  which  information  and  what 
strategies  of  test/model/analyze  should  be  pursued. 

Finally,  costs  and  benefits  can  be  assigned  with  clarity 
to  specific  metrics  and  operators  to  support  CAIV  and 
AoA  studies. 
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4300  GOODFELLOW  BLVD 
ST  LOUIS,  MO  63120-1798 


DISTRIBUTION  LIST 


No.  of  Copies  Organization 

1  CMDR  US  ARMY  AVN  TROOP  CMND 

TECHNOLOGY  APPLCTN  PROG  OFC 
ATTN:  AMSAT  D  TS 

4300  GOODFELLOW  BLVD 
ST  LOUIS,  MO  63120-1798 

1  PRODUCT  MANAGER 

ATTN:  SFAE  AV  LB 

BLDG  105 

4300  GOODFELLOW  BLVD 
ST  LOUIS,  MO  63120-1789 

2  PROJ  MANAGER  CRUSADER 

PEO  FIELD  ARTILLERY  SYS 
ATTN:  SFAE  FAS /SFAE  FAS  CR 

PICATINNY  ARSENAL,  NJ  07806-5000 

1  PROJ  MANAGER  HTV 

PEO  TACTICAL  WHEELED  VEH 
ATTN:  SFAE  TWV  HTV 

WARREN,  MI  48397-5000 

3  PROJECT  MANAGER  BFVS 
PEO  ASM 

ATTN:  SFAE  ASM  BV/ SFAE  ASM  AB 

SFAE  ASM  CV 

WARREN,  MI  48397-5000 

1  PROD  MGR  PALADIN  FAASV 

PEO  FIELD  ARTILLERY  SYS 
ATTN:  SFAE  FAS  PAL 

PICATINNY  ARSENAL,  NJ  07806-5000 

1  PROJ  MGR  MLRS 

PEO  TACTICAL  MISSILES 

ATTN:  SFAE  MSL  ML 

REDSTONE  ARSENAL,  AL  35898-5700 

3  PROJ  MGR  HDSB/PROJ  MGR  HMMWV/PROJ  MGR  LTV 

PEO  TACTICAL  WHEELED  VEH 
ATTN:  SFAE  TWV  HTV/ SFAE  TWV  LTV 

WARREN,  MI  48397-5000 

1  PROJECT  MGR  LW  155 

PEO  FIELD  ARTILLERY  SYS 
ATTN:  SFAE  FAS  J 

PICATINNY  ARSENAL,  NJ  07806-5000 

1  PROJ  MGR  MCD 

PEO  ASM 

ATTN:  SFAE  ASM  MCD 

PICATINNY  ARSENAL,  NJ  07806-5000 
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PRO J  MGR  ARMY  TACMS  BA 

PEO  TACTICAL  MISSILES 

ATTN:  SFAE  MSL  AB 

REDSTONE  ARSENAL,  AL  35898-5650 

PROD  MGR  LB  HELLFIRE 

PEO  TACTICAL  MISSILE 

ATTN:  SFAE  MSL  HD 

REDSTONE  ARSENAL,  AL  35898-5610 

PRO J  MGR  JAVELIN 

PEO  TACTICAL  MISSILES 

ATTN:  SFAE  MSL  AM 

REDSTONE  ARSENAL,  AL  .  35898-5720 

PRO J  MGR  SADARM 

PEO  FIELD  ARTILLERY  SYS 

ATTN:  SFAE  FAS  SD 

PICATINNY  ARSENAL,  NJ  07806-5000 

PRO J  MGR  TANK  MAIN 
ARMAMENT  OFFICERS  TMAS 
PEO  ASM 

ATTN:  SFAE  ASM  TMA 

PICATINNY  ARSENAL,  NJ  07806-5000 

PRO J  MGR  CLOSE  COMBAT 

ANTI  ARMOR  WEAPON  OFFICER 

PEO  TACTICAL  MISSILES 

ATTN:  SFAE  MSL  CC 

REDSTONE  ARSENAL,  AL  35898-5710 

PRO J  MGR  NON  LINE  OF  SIGHT 

COMBINED  ARMS  NLOSCA 

PEO  TACTICAL  MISSILES 

ATTN:  SFAE  MSL  NL 

REDSTONE  ARSENAL,  AL  35898-5793 

PROD  MGR 
USA  ARDEC 
ATTN:  AMCPM  SA 

PICATINNY  ARSENAL,  NJ  07806-5000 

PROJECT  MANAGER 
ARMORED  SYS  INTEGRATION 
PEO  ASM 

ATTN:  SFAE  ASM  AS 
WARREN,  MI  48397-5000 

COMMANDER 

US  ARMY  MATERIEL  CMND 
DEP  CHF  OF  STAFF  FOR  RDE 
ATTN:  AMCRD/AMCRD  IT 
5001  EISENHOWER  AVE 
ALEXANDRIA,  VA  22333-0001 
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COMMANDER 

US  ARMY  MATERIEL  CMND 
PRINCIPAL  DEP  FOR  ACQSTN 
ATTN:  AMCDCG  A/AMCDCG  T 

5001  EISENHOWER  AVE 
ALEXANDRIA,  VA  22333-0001 

COMMANDER 

US  ARMY  ATCOM 

ATTN :  AMSAT  G 

4300  GOODFELLOW  BLVD 

ST  LOUIS,  MO  63120-1798 

COMMANDER 

US  ARMY  AVN  TROOP  CMND 
AVIATION  APPL  TECH  DIR 
ATTN:  AMSAT  R  TV 

FORT  EUSTIS,  VA  23604-5577 

CpMMANDER 

USA  SIM  TRNG  INSTRU  CMND 
ATTN:  AMCP  DIS 
12350  RESEARCH  PKWY 
ORLANDO,  FL  32826-3276 

TECHNICAL  DIRECTOR 
USAARDEC 

ATTN:  SIVICAR  TD 

PICATINNY  ARSENAL,  NJ  07806-5000 

ARMY  ARMAMENTS  RDEC 
ATTN :  SMCAR  ASM 

PICATINNY  ARSENAL,  NJ  07806-5000 

TECHNICAL  DIRECTOR 
USA  AVN  TROOP  CMND 
ATTN:  AMSAT  R  ZT 

4300  GOODFELLOW  BLVD 
ST  LOUIS,  MO..  63120-1798 

TECHNICAL  DIRECTOR 
USA  MISSILE  CMND  RDE  CTR 
ATTN:  AMSMI  RD 

REDSTONE  ARSENAL,  AL  35898-5240 
COMMANDER 

USA  TANK  AUTO  CMND  RDEC 

ATTN:  AMSTA  TR  D/AMSTA  RE  D/AMSTA  TR  R 

WARREN,  MI  48397-5000 
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ARMY  RESEARCH  LABORATORY 
SURV  LETH  ANALYSIS  DIR 
ATTN:  AMISRL  SL 

AMSRL  SL  CA 
AMSRL  SL  E 
AMSRL  SL  EA 
AMSRL  SL  EV 
AMSRL  SL  EP 
AMSRL  SL  EG 
AMSRL  SL  EU 
AMSRL  W 

WHITE  SANDS  MISSILE  RANGE,  NM  88002-5513 

ARMY  RESEARCH  LABORATORY 
SURV.  LETH  ANALYSIS  DIR 
ATTN:  AMSRL  SL  EL 

FT  MONMOUTH,  NJ  07703-5601 

DIRECTOR 

A£MY  FIELD  ARTILLERY  SCHOOL 
ATTN:  ATSF  FSC  3A 

ATSF  CN 
ATSF  RMS 

FT  SILL,  OK  73503-5600 
DIRECTOR 

ARMY  INFANTRY  SCHOOL 
ATTN:  AT ZB  FS 

ATZB  BV 

FT  BENNING,  GA  31905-5400 
DIRECTOR 

US  ARMY  AIR  DEFENSE 
ARTILLERY  SCHOOL 
ATTN:  ATS  A  TSM  TM/ATSA  CD 

FT  BLISS,  TX  79916-0002 

DIRECTOR 

ARMY  ARMOR  SCHOOL 

ATTN:  ATZK/ATZK  SM/ATZK  XL 

BLDG  1109 

FT  KNOX,  KY  40121-5201 

COMMANDER 
US  ARMY  ARMOR  CTR 

ATTN:  ATZK/ATZK  MW/ATZK  SM/ATZK  XL. 

FT  KNOX,  KY  40121-5000 

DIRECTOR 

ARMY  AVIATION  CTR 

ATTN:  ATZQ  TSM  C/ATZQ  TSM  LB 

FT  RUCKER,  AL  36362-5010 
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DIR  ARMY  ENGINEER  CTR 
AND  FT  LEONARD  WOOD 
ATTN:  ATSE  TSM 

FT  LEONARD  WOOD,  MO  65473-5000 
DIRECTOR 

ARMY  TRANSPORTATION  SCH 

ATTN:  ATZF  TW 

FT  EUSTI S ,  VA  23604-5457 

DIRECTOR 

ARMY  TRADOC  ANALYSIS  CTR 

ATTN:  ATRC  W/ATRC-WMB/ATRC  WS 

WHITE  SANDS  MISSILE  RANGE,  NM  88002-5502 

COMMANDANT 

US  ARMY  INFANTRY  SCH 

ATTN:  ATSH  CD 

FT  BENNING,  GA  31905-5403 

COMMANDER 

USA  FIELD  ARTILLERY  CTR 

ATTN:  ATSF  CD 

FT  SILL,  OK  73503-5000 

COMMANDANT 

USA  ARMOR  CTR 

ATTN:  AT SB  CD 

FT  KNOX,  KY  40121-5000 

COMMANDANT 

US  ARMY  AVIATION  CTR 
ATTN:  ATZQ  CD 

FT  RUCKER,  AL  36362-5000 

DEPT  OF  NAVY 

DEP  DIR  NAVY  T&E  TECH  REQ 
PENTAGON  RM  5C686 
WASHINGTON,  DC  20350 

OFFICE  OF  NAVAL  RESEARCH 
TECH  AREA  MGR  SURF  AERO 
WEAPONRY  CODE  35 
800  N  QUINCY  ST 
ARLINGTON,  VA  22217-5660 

NAVAL  AIR  OFFICERS  CMD  HQ  *  ' 

1421  JEFFERSON  DAVIS  HIGHWAY 
ARLINGTON,  VA  22243 

NAVAL  SURFACE  WARFARE  CTR 
CARDEROCK  DIVISION 
BETHESDA,  MD  20084-5000 
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NAVAL  SURFACE  WARFARE  CTR 
DAHLGREN  DIVISION 
ATTN:  G24 
17320  DAHLGREN  RD 
DAHLGREN,  VA  22448-5100 

NAVAL  SAFETY  CTR 
CODE  471 

NORFOLK,  VA  23501 

NAVAL  POSTGRADUATE  SCHOOL 
CODE  AABP 

MONTEREY,  CA  93943-5000 

NAWC 

526EOOD 

1  ADMINSTRATION  CIRCLE 
CHINA  LAKE,  CA  93555-6100 

DHP  DIR 

HQ  US  AIR  FORCE  T&E 
1650  AIR  FORCE  PENTAGON 
WASHINGTON,  DC  20330-1650 

USAF  PHILLIPS  LABORATORY 
ATTN:  PIL  WS 
3550  ABERDEEN  AVE  SE 
KIRKLAND  AFB,  NM  87117-5576 

USAF  WRIGHT  LABORATORY 
ATTN:  WL  FIVS 

WRIGHT  PAT  AFB,  OH  45433-6553 

USAF  WRIGHT  LABORATORY 
ARMAMENT  DIRECTORATE 
ATTN:  WIL  MN/WIL  MNP 

101  EGLIN  BVLD  SUITE  302 
EGLIN  AFB,  FL  32542-6810 

USAF  WRIGHT  LABORATORY 
1901  TENTH  ST 

WRIGHT  PAT  AFB,  OH  45433-7605 

USAF  ASC  XRA 
BLDG  16 

2275D  ST  SUITE  10 

WRIGHT  PATTERSON  AFB,  OH  45433-7227 

HQ  USAF  SPECIAL  OPERATIONS  CMD 
AFSOC  LGMW 
100  BARLEY  ST 
HURLBURT  FIELD,  FL  32536 
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US  AF  OPER  TEST  &  EVAL  CMD 
ATTN:  AFOCTE  CN 

8500  GIBSON  BVLD  SE 
KIRKLAND  AFB,  NM  87117-5558 

USA  AVIATION  LOGISTICS  SCHOOL 
AVIATION  TRADES  TRAINING 
BLDG  2715F 

FT  EUSTIS,  VA  23604-5439 

ARMY  RESEARCH  OFFICE 
MATH  AND  CS  DIV 
PO  BOX  1221 

RESEARCH  TRIANGLE  PARK,  NC  27709-2211 
US  GAO 

DIR  PROG  EVAL  PHYS  SYS 
441  G  STREET  NW 
RM  4062 

WASHINGTON,.  DC  20548 

LAWRENCE  LIVERMORE  NATIONAL  LABORATORY 
PO  BOX  808  L159 
LIVERMORE,  CA  94551 

LOS  ALAMOS  NATIONAL  LAB 
CONV  WEAPON  TECHNOLOGIES 
PO  BOX  1663 
MAIL  STOP  AL  33 
LOS  ALAMOS,  NM  87545 

CMDR  USA  OPTEC 
4501  FORD  AVE 
CSTE  ZT 

PARK  CENTER  IV 
ALEXANDRIA,  VA  22302-1458 

U.S.  SPACE  &  STRATEGIC  DEFENSE  CMD 
ATTN:  HELSTFCSSD  HID 

WHITE  SANDS  MISSILE  RANGE,  NM  88002-5148 

USA  TEXCOM 
ATTN:  CSTE  TTD 
TECHNICAL  DIR 
FT  HOOD,  TX  76544-5065 

NWSCDL  *  * 

JTCG  ME  PROGRAM  OFFICE  G202 
BLDG  221  RM  245 
DAHLGREN,  VA  22448-5100 

US  MILITARY  ACADEMY 
DEPT  OF  MATH  SCI 
THAYER  HALL 

WEST  POINT,  NY  10996-1786 
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DR  MARTHA  K  NELSON' 

DEPARTMENT  OF  BUSINESS  ADMIN 
FRANKLIN  AND  MARSHALL  COLLEGE 
PO  BOX  3003 

LANCASTER,  PA  17604-3003 

DR  ED  WEGMAN 
GEORGE  MASON  UNIVERSITY 
CTR  COMPUTATIONAL  STATISTICS 
FAIRFAX,  VA  22030 

DEPT  OF  INDUSTRIAL  ENGINEERING 
UNIV  OF  WISCONSIN  MADISON 
ATTN:  S  M  ROBINSON 

1513  UNIVERSITY  AVE 
MADISON,  WI  53706-1572 

AD  PA 

2101  WILSON  BLVD  SUITE  400 
ARLINGTON,  VA  22201-3061 

r 

APPLIED  RES  ASSOCIATES  INC 
4300  SAN  MATEO  BLVD 
SUITE  A220 

ALBUQUERQUE,  NM  87110 

APPLIED  RES  ASSOCIATES  INC 
219  W  BEL  AVE  SUITE  5 
ABERBEEN,  MD  21001 

AS  I  TEST  CONSULTANT 
825  NORTH  DOWNS  SUITE 
RIDGECREST,  CA  93555 

AS I  OFFICERS  INTL 
56  IVERNESS  DRIVE  E 
SUITE  260 

ENGLEWOOD,  CO  80112-5114 

BOOZ  ALLEN  &  HAMILTON  INC 
WIL  FIVS  SURVIAC 
2130  EIGHTH  ST  SUITE  1 
WRIGHT  PAT  AFB,  OH  45433-7542 

BRIGS  COMPANY 

2668  PETERSBOROUGH  ST 

HERNDON,  VA  20171 

DIRECTED  TECHNOLOGIES  INC 
4001  N  FAIRFAX  DRIVE 
SUITE  775 

ARLINGTON,  VA  22203 
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DUAL  INC 

2101  WILSON  BVLD 
SUITE  600 

ARLINGTON,  VA  22201-3078 

GEORGIA  INSTITUTE  OF  TECH 
T&E  RESEARCH  &  EDUCATION  CTR 
ML  CD  0840 

CRB  BUILDING  ROOM  631 
400  TENTH  STREET 
ATLANTA,  GA  3  0318 

HICKS  AND  ASSOCIATES  INC 
1710  GOODRIDGE  DRIVE 
SUITE  1300 
MCLEAN,  VA  22044 

INSTITUTE  FOR  DEFENSE  ANALYSIS 
1801  N  BEAUREGARD  STREET 
ALEXANDRIA,  VA  22311-1772 

INSTITUTE  FOR  DEFENSE  ANALYSIS 
3807  BENT  BRANCH  RD 
FALLS  CHURCH,  VA  22041 

K&B  ENGINEERING  ASSOCIATES 
6109  G  ARLINGTON  BVLD 
FALLS  CHURCH,  VA  22044 

LOCKHEED  MARTIN 
VOUGHT  OFFICERS  CORP 
PO  BOX  650003 
MAILSTOP  EM  36 
DALLAS,  TX  75265-0003 

LOGICON  RDA 

2100  WASHINGTON  BVLD 

ARLINGTON,  VA  22204 

LOGISTICS  MANAGEMENT  INSTITUTE 
2000  CORPORATE  RIDGE 
MCLEAN,  VA  22103-7805 

QUANTUM  RESEARCH 
INTERNATIONAL 

CRYSTAL  SQUARE  5  SUITE  703 
1755  JEFFERSON  DAVIS  HIGHWAY 
ARLINGTON,  VA  22202-3509 

QUESTECHINC 

901  N  STUART  STREET 

SUITE  605 

ARLINGTON,  VA  22203 
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DISTRIBUTION  LIST 
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1  RAYTHEON  COMPANY 

EXECUTIVE  OFFICES 
141  SPRING  STREET 
LEXINGTON,  MA  02173 

1  ROCKWELL  INTERNATIONAL 

ROCKETDYNE  DIVISION 
6633  CANOGA  AVE 
EB58 

CANOGA  PARK,  CA  91309 

1  SAIC 

2301  YALE  BVLD  SE 
SUITE  E 

ALBUQUERQUE,  NM  87106 

1  SAIC 

TEST  EVAL  &  ANAL  OPERATIONS 
8301  GREENSBORO  DRIVE 
SUITE  460 
PO  BOX  50132 
MCLEAN,  VA  22102 

1  THE  SURVICE  ENGINEERING  CO 

1003  OLD  PHILADELPHIA  RD 
SUITE  3 

ABERDEEN,  MD  21001 

1  BOB  BARR 

18605  CABIN  RD 
TRIANGLE,  VA  22172-1505 

1  ROBERT  DEITZ 

PERKINS  COIE 
607  FOURTEENTH  ST  NW 
WASHINGTON,  DC  20005-2011 
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